Computing tips for awesome research and an easy life

Robert Turner, University of Sheffield RSE Team September, 2021

Acknowledgements

Contains elements from Reproducible Research Data and Project Management in R, by Anna Krystalli and from Methods in Research Software Engineering by David Wilby.

About me

Mix of software engineering and research experience.

RSE Team

13 RSEs, 35 projects / year worth ~£11m total

This presentation is flawed

Focusses on what to do, not how to do it.

In this session…

Discussion

What are the characteristics of well engineered research software?

Link to interactive doc

“Good” research software

Version control
Automated tests
Controlled execution environment
Documentation
Parameterised

https://joss.readthedocs.io/en/latest/review_checklist.html

Data Management

Data Management Plan

Start early. Make an RDM plan before collecting data.
Anticipate data products as part of your thesis outputs.
Think about what technologies to use.

Own your data

Act as though every short term study will become a long term one @tomjwebb. Needs to be reproducible in 3, 20, 100 yrs
— Oceans Initiative (@oceansresearch) January 16, 2015

Take initiative & responsibility. Think long term.

Spreadsheets?

@tomjwebb stay away from excel at all costs?
— Timothée Poisot (@tpoi) January 16, 2015

Do you agree?

Excel

THRILLED by this announcement by the Human Gene Nomenclature Committee. pic.twitter.com/BqLIOMm69d
— Janna Hutz (@jannahutz) August 4, 2020

But good for data viewing / entry, sometimes, perhaps…

Databases

@tomjwebb Entering via a database management system (e.g., Access, Filemaker) can make entry easier & help prevent data entry errors @tpoi
— Ethan White (@ethanwhite) January 16, 2015

@ethanwhite +1 Enforcing data types, options from selection etc, just some useful things a DB gives you, if you turn them on @tomjwebb @tpoi
— Gavin Simpson (@ucfagls) January 16, 2015

@tomjwebb it also prevents a lot of different bad practices. It is possible to do some of this in Excel. @tpoi
— Ethan White (@ethanwhite) January 16, 2015

Have a look at the Data Carpentry SQL for Ecology lesson

Data formats

.csv: comma separated values.
.tsv: tab separated values.
.txt: no formatting specified.

@tomjwebb It has to be interoperability/openness - can I read your data with whatever I use, without having to convert it?
— Paul Swaddle (@paul_swaddle) January 16, 2015

more unusual formats will need instructions on use.

Ensure data is machine readable